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Several  different  models  of  parallelism  have  been  discussed  so  far  at 
this  conference  — Petri  nets  and  Vector  Addition  Systems,  to  name  two. 

These  models  have  been  around  for  some  years  and  we  are  all  familar 
with  them  (though  they  may  not  be  fully  understood  as  yet).  By  contrast, 
we  will  present  a new  model  that  introduces  the  idea  of  bounded  asynchronous  ' 
delay.  This  idea  provids  a useful  tool  for  studying  both  synchronism  and 
purely  event-driven  asynchronism  by  Introducing  a spectrum  of  approximating 
models  to  fill  the  gap  between  these  two  extremes. 

Our  goal  is  two-fold:  ^ 

(1)  to  introduce  our  model  of  bounded  delay,  and 

(2)  to  point  out  some  of  the  subtleties  of  asynchronism  that  are 
elucidated  by  our  model. 

Thus,  we  hope  to  interest  you  in  bounded  asynchronism,  generally,  and  our 
model  specifically  as  well  as  to  point  out  some  of  the  difficulties  of 
event-driven  asynchronism. 

1.  Preliminaries 

Our  model  addresses  questions  of  asynchronism  with  bounded  delay  and 
there  do  not  seem  to  be  any  theoretical  treatments  of  this  phenomenon 
in  the  literature.  What  we  have  done  13  pretty  much  to  generalize  the 
cellular  array  characterization  of  Smith  and  others  [1),  where  we  consider 
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of  n finite  state  machines  capable  of  communicating  with  their  neighbors. 

In  the  classic  treatment  of  these  devices,  time  is  measured  In  discrete 
steps  and  the  evaluation  rule  is  "all  machines  capable  of  firing  at  a 
given  step,  do  so."  We  introduce  asynchronism  by  relaxing  this  rule  to 
"some  machine(s)  capable  of  firing  at  a given  step  do  so.  Thus,  a device 
can  fire  "at  will,"  so  to  speak,  and  thus  the  concept  of  machines  operating 
at  different  rates  can  be  characterized. 

All  of  the  usual  questions  asked  of  the  synchronous  parallel  arrays  can 
be  asked  about  these  asynchronous  arrays,  such  as  synchronization,  recognition 
capabilities  etc.  But  an  immediate  observation  is  that  in  this  asynchronous 
model  some  device,  capable  of  firing,  may  postpone  doing  so  for  an  unbounded 
or  possible  infinite  number  of  steps. 

Although  exact  response  times  are  not  always  known,  it  is  usually  the  case 
that  some  information  is  available  about  the  relative  response  times.  In 
such  cases  we  believe  that  the  information  should  be  used.  Hence  we 
Introduce  the  concept  of  delay , an  upper  bound  on  the  response  time  of  any 
device.  As  a consequence,  we  get  sharper  and  more  quantitative  results  as 
well  as  providing  a convenient  means  of  comparing  synchronous  and  event-driven 
asynchronous  behavior. 

Delay  is  a fixed,  nonnegative  integer  upper  bound  on  the  number  of  steps 
a device  capable  of  firing  is  allowed  to  postpone  that  action.  Hence,  when 
D ■ 0,  no  postponesment  is  allowed  and  the  system  operates  synchronously. 

Mote  the  difference  between  delay  and  frequency  of  firing.  In  particular, 

• device  which  is  always  capable  of  firing  must  fire,  at  worst  every  D + 1 
steps.  When  D tends  to  infinity  the  system  tends  to  act  totally  as  an 
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event-driven  asynchronous  system.  Thus,  we  have  a whole  spectrum  of 
execution  disciplines  between  the  extreme  points. 

Before  actually  Introducing  the  model,  it  may  be  useful  to  mention  the 
difference  between  bounded  asynchronism  and  nondelerminism.  The  distinction 

Is,  of  course,  between  "what"  to  do  and  "when"  to  do  it.  Hence,  there  are 

.*  • 

really  four  possible  models  involving  these  two  concepts: 


deterministic  synchronous 
nondeterminlstic  synchronous 
deterministic  asynchronous 
nondeterminlstic  asynchronous 


To  underscore  the  differences,  consider  the  three  element  linear  array 

1 -£_1  GH 
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and  two  types  of  state  diagrams: 
deterministic: 


nondeterminlstic: 


©- 

©- 


u 


© 

© 

© 


* » 


HR392 
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where  u means  that  the  transition  is  legal  regardless  of  the  states  of 
the  neighbors.  Then  we  can  construct  a table: 


•ynchronous  (D  - 0)  asynchronous  CB  2) 


deterministic 

/ computation  seq:  1 

/computation  seq: 

13 

/different  outputs:  1 

/different  outputs: 

1 

nondeterministic 

/computation  seq:  27 

/computation  seq: 

351 

/different  outputs:  27 

/different  outputs: 

27 

The  computation  sequences  possible  are  compared  with  the  number  of 
possible  outputs  and  we  see  that  for  this  simple  system,  the  asynchronism 
influences  how  the  output  is  arrived  at  (computation  sequences)  but  not 
the  output  Itself.  Note  that  the  tabic  is  for  some  D i 2 and  that  if 
D * 0 were  assumed,  both  columns  would  be  the  same. 

This  property,  which  we  might  call  transparency , where  the  asynchronism 
influences  only  the  way  the  result  is  obtained  but  not  the  result  itself, 
does  not  necessarily  always  hold.  In  particular,  there  arc  deterministic 
machines  which,  when  run  asynchronously,  give  several  different  outputs. 

This  is  because  a device,  capable  of  changing  from  state  a to  state  b , can 
postpone  that  transition  and  then,  as  a result  of  a state  change  by  one  of 
its  neighbors,  it  may  be  capable  of  a transition  from  a to  c.  Consequently, 
nondete’-tnlnism  has  apparently  been  Introduced  by  the  asynchronism!  This 
phenomenon,  which  might  be  called  surreptitious  nondeterminism,  is  poorly 
understood  and  preliminary  indications  are  that  it  leads  to  a different 
class  of  systems.  Our  Interest  here  will  be  to  achieve  transparency  for 
the  deterministic  asynchronous  case,  since  this  Is  a broad  and  more 
manageable  class  of  systems. 
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2.  The  Model 


Since  working  with  finite  state  machines  is  cumbersome,  we  have  developed 
a rewriting  system  which  generalizes  the  standard  finite  state  machine 
model. 

Definition:  An  asynchronous  grarunar  G ■ (£,  P)  is  a finite  alphabet 
£ and  a finite  set  of  productions  of  the  form  a -*  £ where 


(1) 

* 

a,e  C I 

(symbols  represent  states) 

(il) 

|a|  " IM 

(length  preserving) 

(iii) 

a t $ 

(no  "idling"  productions) 

A set  of  devices  in  various  states  and  with  various  interconnections 
is,  thus,  represented  by  a word  in  Ln.  A symbol  capable  of  being 
cnanged  by  a production  application  is  said  to  he  active.  A computation 
is  described  by  defining  [ — ■. 

Definition:  Let  x • x,...x  . y ■ y...'.y  where  x.y  e En.  Then 

1 o ' '1  n ' 


if  x y y and  if  the  i1*1  position  of  x changes  (i.e.  i y^)  then  there 
exists  a production  a^...Oj...a^  <♦  such  that 


(1)  a^. . .Oj. . . matches  some  context  surrounding 

(i.e.  3^  > x 


i-J+1 


•••  •••  • • Oj  • • • 


and 


(ii)  the  changes  implied  by  the  productions  obtain  and  are 
consistent  (i.e.  o#  1 =>  y ^ for 

• •1,  2,  Mi)  k). 


o. 


The  first  requirement  guarantees  that  the  transitions  performed  are 
legal  according  to  the  production  set  and  the  second  requirement  makes 
certain  that  all  changes  take  place  and  are  not  contradictory. 

% 

As  an  example , let  £ « {a,  b,  c,  d)  and  P - 

{ab  ac 
be  ■*  bd 
ab  * dc) 

then  the  legal  transitions  would  be,  for  abc, 


1. 

abc 

acc 

ab 

ac  applied 

2. 

abc 

I — abd 

be 

bd  applied 

3. 

abc 

| — dec 

ab 

dc  applied 

4. 

abc 

| — acd 

ab  -*•  ac 

and 

be 

bd  applied 

5. 

abc 

| — ded 

either  ab 

-► 

dc 

and  be  -*  bd 

applied 

or 

all 

• * 

Note  that  in  the  fourth  case  the  two  productions  do  overlap  but  the 
changes  arc  made  consistently  and  so  this  is  allowed  by  the  definition. 
Also,  in  the  final  case  (and  3 as  well)  it  is  ambiguous  just  exactly 
what  productions  applied  since  the  production  ab  -+  dc  subsumes  the 
production  ab  ac.  Another  point  to  note  is  that  multiple  changes 
are  allowed  (ab  •+  dc).  Clearly,  very  complex  behavior  can  be  described 
by  asynchronous  grammars. 

The  reflexive  transitive  closure  ()-£■ ) of  }—  is  defined  in  the  usual 
way,  but,  as  the  following  definition  indicates,  some  sequences  nay  not 
qualify  as  acceptable  computations.  (Notatlonally , superscripts  are  used 
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for  elements  of  a sequence  and  subscripts  are  used  for  coordinates.) 

Definition;  Let  x°,  x1,  ...  be  in  In,  G = (£,P)  be  an  asynchronous 
grammar  and  D £ 0 be  an  integer.  A D- computation  is  a sequence 
J>  1 

X t X f • • • 


such  that 


(!)  V J i 0,  x*  |-  xJ+1 


and 


(ii)  i|]  i.J 

is  active  for  all  k = 1,  2,  ...»  D+l. 


such  that  x^  “ 


i-t-k 

x^J  and  coordinate  i 


Thus,  a D-conputation  is  a sequence  of  legal  transitions  defined  by  an 

asynchronous  grammar  where  no  postion  postpones  firing  longer  than  D 

% 

»teps.  When  D **  G,  (i.e.  no  postponement),  the  I>-co=?utatlon  is  said 

to  be  synchronous  — i.e.  when  a postion  becomes  active  at  the  j1*1  step 

s t 

it  fires  at  the  j+1  6tep.  A D-cogputation 


halts  when 
take  place. 


^ x c Ln 


0 m 

x x 


6uch  that  xm  |—  x,  i.e.  when  no  further  changes 


As  an  example,  consider  £ - {*,a,b,c}  and  P 


{*a 

•* 

baa 

ac 


*b 
c* 
bba 
cc  ) 


then  the  0-computation  on  *aaaa*  is 
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*aaaa*  }—  *baac*  h~  *bbcc* 
while  for  D 2 3,  the  D-coraputations  are 

*aaaa*  | — *bbcc* 

*aaaa*  I — *bbbc* 

*aaaa*  J—  *bccc* 

*aaaa*  |— * *cccc* 

where  each  computation  has  been  forced  to  a halting  state.  Now,  clearly, 
this  particular  grammar  on  *aaaa*  Is  not  transparent  although  it  Is 
deterministic  in  the  sense  of  finite  state  machines.  Note  where  the 
surreptitious  nondeterrinism  crept  lu. 

« • • beac  ...  ] ...  bacc  ... 

can  go  only  can  go  only 

to  b -to  c 

(Note  that  in  the  degenerate  case,  the  grarmar  isn't  even  deterministic 

I 

since  *a*  | — *b*  and  *a*  [—  *c*  are  legal.) 

There  are  several  ways  to  restrict  asynchronous  grammars  to  only  those 
which  lead  to  transparent  D-computations.  One  such  method  to  avoid 

i 

the  interference  inherent  in  the  previous  example  will  now  be  considered. 


Definition : An  asynchronous  grammar  C •*  (E,P)  is  interference- free 
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if  Vp,p'  and  t 


p a al’  ’ *at+l*  "a 


* Bl*,,Bt+l*’*Bk  ' 


a'  ..a'  PI...B! 

11  11 


and  a_  - oJ_  i - 1,2,...,  min(l,  k-t)  implies 


tHi 


t+i 


t+i 


Informally,  an  asynchronous  grammar  is  interference-free  if  whenever 
two  productions  overlap,  the  overlapping  portion  is  unchanged  in  both 
productions.  Thus 

p:  a ...  bxyz  -*■  e . . . fxyz 

p' : xyzc...  d v 

could  be  productions  of  an  interference-free  grammar. 


How  returning  to  the  grammar  that  suggested  this  restriction,  ve  observe 
that  there  is  an  interference-free  asynchronous  gre—mar  that  is  trans- 
parent and  equivalent  to  the  former  grammar  for  O-computations  on 
*ak*  for  all  k * 2.  Specifically, 

E “ {*,  a,  b,  c,  b',  c',  b,  c } 

P - {*a  *c' 

c'a  cb' 

b'a  be1 

C1  * c* 

b1*  + b* 

. cb  cB 

cB  be ). 


and 
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This  grammar  is  clearly  interference  free  since  the  only  overlaps 
leave  the  overlapping  portions  unchanged: 


and 


c'*  + c* 
*a  *►  *c' 


b’*  + b* 


The  grammar  is  also  substantially  more  complicated  than  the  previous 
one  as  can  be  seen  in  the  following  0-conputation: 

*aaaa*  | — *c’aaa* 

I — *cb 'aa* 

| — *cbc'a* 

I — *cScb'* 

| — *bccb* 

I — *bccS* 

| — *bcbc* 

I *bcbc* 

| — *bbcc* 

Note  that  the  strategy  Is  to  first  initialize  the  string  with  b's 
and  c's  (half  of  each)  and  then  interchange  adjacent  pairs  until  they 
propagate  to  their  respective  ends. 

Rather  than  trying  to  argue  directly  that  this  interference-free 
grammar  is  in  fact  transparent,  we  appeal  to  the  following  theorem. 

The or on  (Transparency  for  interference-free  grammars) 
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Let  G - (£ ,P)  be  an  Interference  free  asynchronous  grammar  such  that 

x"  Is  a halting  O-computation,  then  \/d  > 0,  for  every 

D^-computatlon 

(1)  3q  such  that  x°,  ....  xq  is  a halting  D- computation,  and 
(ii)  xq  ■ xm. 

This  can  be  proved  in  several  ways.  A direct  proof  would  argue  that 
for  each  position  i in  x®,  the  sequence  of  Q-computatlon  state  changes 
that  take  place  also  occur,  in  the  same  order,  for  any  D-coraputation. 

Thus  we  have  a characterization  of  one  class  of  asynchronous  grammars 
which  is  transparent.  Note  that  the  intcrfcrence-f rec  grammar  just 
presented  allowed  multiple  changes.  This  is  not  really  faithful  to 
our  original  objective  where  the  machines  are  to  fire  indlvlduallv.  An 
alternative  characterization  has  been  developed  where  the  single  change 
property  does  hold.  This  alternative  model  and  results  concerning  its 
timing  characteristics,  recognition  capabilities  and  synchronization 
capabilities  is  presented  in  the  proceedings  of  the  IEEE  symposium  on 
the  Foundations  of  Computer  Science  [2]. 

3.  Decidability 

Next  we  consider  some  decidability  questions.  Clearly,  since  the  pro- 
ductions are  length  preserving  and  thus  there  is  a bound  on  the  number 
of  states  from  any  initial  input  x°,  it  follows  that  the  halting 
problem  for  any  given  input  x®  is  decidable,  for  a given  delay  D. 
Moreover,  because  Smith  [1]  has  embedded  a Turing  machine  in  his  model 
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(which  is  contained  in  ours  as  a special  case)  , it  follows  that  it  is  not 
decidable  whether,  for  a given  D,  an  asynchronous  grammar  halts  for 
•11  Inputs.  Also,  the  reachability  problem  is  decidable  for  a £iven 
•synchronous  grammar  G,  delay  D,  input  and  reachability  configu- 

ration xr. 

Those  are  the  usual  questions  one  asks  about  parallel  systems.  But 
because  these  are  asynchronous  systems  too,  we  can  ask  questions  about 
delay.  For  example,  since  increasing  the  value  of  D often  increases 
the  reachable  states,  we  can  ask  the  D-reachability  question. 

Problem:  (D-reachability)  Given  an  asynchronous  grammar  G,  initial 

input  x^  and  reachability  configuration  xr,  what  is  the  least  delay 
D,  if  any,  such  that  x®,  ...»  xr  is  a D-conputation? 

Thus,  if  C *=  (*a  -*•  *b 

a*  -*  c* 
baa  bba 
•c  cc) 

and  x^  ■ *aaaa*  and  xr  ■ *cccc*,  the  D-rcachability  number  is 
2.  That  is,  for  D 5 1,  this  configuration  cannot  be  reached  from 
*aaaa*.  Note  that  the  D-rcachability  number  for  xr  •»  *bbbb*  is  •, 
i.e.  no  matter  how  large.  D is  this  configuarion  cannot  be  obtained 
(due  to  the  third  production). 

The  D-reachablllty  problem  is  decidable. 

The  argument  here  is  based  on  an  enumeration  af  all  reachable  states 


for  successively  larger  D.  The  enumeration  stops  when  D is  large 
enough  so  that  no  transition  fires  due: to  the  expiration  of  the  delay. 
This  value  is  essentially  the  longest  acyclic  sequence  of  single  pro- 
duction applications  that  can  take  place  independent  of  some  pending 
transition. 

Another  variant  on  the  reachability  question  suggested  by  delay  is 
based  on  the  concept  of  duration. 


Definition • Let  x®,  x*t  ...  he  a D-conputatiow  for  uti  asynchronous 
grammar  G on  input  x®.  The  duration  6(i,J)  of  position  J in 
configuration  1 is 


i.J) 


^k+l 


If  Xj  is  not  active 
* 

if  Xj  is  active  and  k is  the  largest 


value  such  that  x 


i-k 


Xj  * 'is  active  for  all  values 

t * Op  ....  k. 


•,nd 


Hence,  the  duration  5(i,J)  gives  the  number  of  consequtive  steps 
that  the  J1*1  position  has  been  waiting  to  fire  as  of  the  i**1  step  in 
the  D-computatlon. 


Problem:  (Duration  reachability)  Given  an  asynchronous  grammar  C, 

initial  configuration  x®,  reachability  configuration  xr  and  n 
Integers  k^,  ...,  k^,  is  it  decidable  whether  or  not  there  exists  a 


D such  that 
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Is  a D-computation  with  6(r,j)  - k^? 

Hence,  we  not  only  want  to  know  if  we  can  reach  a particular  configura- 
tion, but  also  do  the  positions  have  a particular  duration  value. 

The  duration— reachability  problem  is  decidable.  The  argument  uses  the 
same  enumeration  as  for  the  D— reachability  problem  but  with  the  added 
constraint  of  a complex  search  to  determine  if  the  proper  duration 
values  hold. 

4.  Timings 

Finally,  note  should  be  made  of  the  fact  that  although  asynchronous 
grammars  operating  asynchronously  are  weaker  in  certain  respects  than 
their  synchronous  counterparts,  the  reason  is  not  due  to  their  timings. 
(The  reason  is  in  fact  their  inability  to  solve  problems  such  as  the 
"firing  squad  synchronization  problem,"  see  [2].)  Indeed,  one  gets  the 

impression  from  our  discussion  that  as  D Increases  and  the  length 
of  the  computation  sequences  increase,  the  execution  tine  of  the  parallel 
system  degrades.  Our  experience,  of  course,  tells  us  that  asynchronism 
should  probably  improve  rather  than  degrade  the  performance.  Reconciling 
these  two  observations  is  easy:  the  quantity  D + 1 always  represents 

one  unit  of  physical  time  and  consequently  an  increase  in  D merely 
means  that  time  is  being  partitioned  into  finer  and  finer  units.  Hence, 
given  timings  for  the  various  transitions,  the  synchronous  case  runs  at 
a rate  equal  to  the  time  of  the  longest  transition.  The  asynchronous 
transitions  will  vary  in  cost;  each  being  charged  according  to  the 
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actual  usage.  Hence  even  though  the  computation  sequences  are  longer 
In  terms  of  the  nunber  of  transitions  in  the  asynchronous  case,  the  actual 
cost  need  not  be  larger. 

In  conclusion,  we  emphasize  that  the  important  property  of  this  model  is 
that  delay  is  treated  parametrically.  Thus,  a single  system-may  be  studied 
with  a synchronous  as  well  as  an  asynchronous  exectuion  behavior  within 
a single  model  merely  by  changing  the  quantity  of  the  delay.  Comparison  of 
the  two  types  of  execution  is,  therefore,  quite  convenient.  It  is  hoped 
that  this  introduction  has  presented  sufficient  motivation  to  Interest 
others  in  taking  a similar  approach  with  other  models  of  parallel  computation. 
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